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Abstract 

Inspired by successful biological collective decision mechanisms such as honey bees searching for 
a new colony or the collective navigation of fish schools, we consider a mean field games (MFG)-like 
scenario where a large number of agents have to make a choice among a set of different potential target 
destinations. Each individual both influences and is influenced by the group’s decision, as represented 
by the mean trajectory of all agents. The model can be interpreted as a stylized version of opinion 
crystallization in an election for example. The agents’ biases are dictated first by their initial spatial 
position and, in a subsequent generalization of the model, by a combination of initial position and a priori 
individual preference. The agents have linear dynamics and are coupled through a modified form of 
quadratic cost. Fixed point based finite population equilibrium conditions are identified and associated 
existence conditions are established. In general multiple equilibria may exist and the agents need to 
know all initial conditions to compute them precisely. However, as the number of agents increases 
sufficiently, we show that (i) the computed fixed point equilibria qualify as epsilon Nash equilibria, (ii) 
agents no longer require all initial conditions to compute the equilibria but rather can do so based on a 
representative probability distribution of these conditions now viewed as random variables. Numerical 
results are reported. 
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I. Introduction 

Collective decision making is a common phenomenon in social structures ranging from animal 
populations Q, Q to human societies 0. Examples include honey bees searching for a new 
colony 0. Q. the navigation of fish schools 0. 0. or quorum sensing p0| . Collective 
decisions involve dynamic “microscopic-macroscopic” or “individual-social” interactions. On 
the one hand, individual choices are socially influenced, that is influenced by the behavior of 
the group. On the other hand, the collective behavior itself results from aggregating individual 
choices. In elections for example, an interplay between individual interests and collective opinion 
swings leads to the crystallization of final decisions [[^, pT|, p^. 


“Homing” optimal control problems, first introduced by Whittle and Gait in [13| and studied 
later in for example, are concerned with a single agent trying to reach one of multiple 

predefined final states. Here we consider a similar fundamental issue but in a multi-agent setting. 
A large number of agents initially spread out in need to move within a finite time horizon 
to one of multiple possible home or target destinations. They must do so while trying to remain 
tightly grouped and expending as little control effort as possible. Our goal is to model situations 
in which the choice made by each agent regarding which destination to reach both influences 
and depends on the behavior of the population. For example, when honey bees determine their 
next site to establish a colony, they must make a choice between different alternatives based 
on the information provided by scouts, who are themselves part of the group. Even though 
certain colonies can be easier to reach and are more attractive for some bees, following the 
majority is still a priority to enhance the foraging ability. Similarly, in a navigation situation for 
a collection of micro robots exploring an unknown terrain, remaining grouped may be necessary 


for achieving coordinated collective tasks [18|-|21|. In animal collective navigation, discrete 
choices must be made regarding the route to take, but at the same time staying with the group 
offers better protection against predators. Finally, our model may be an abstract representation 
of opinion crystallization in an election where (i) relative distances measure current differences 
of opinions, (ii) individuals are sensitive to collective opinion swings, and (iii) a choice must be 
made before a finite deadline @, [ fTT| , [ |T^ . 

A related topic in economics is discrete choice models where an agent makes a choice between 


multiple alternatives such as mode of transportation [22|, entry and withdrawal from the labor 
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market, residential loeation p3| , or a physieian p4| . In many cireumstanees, these individual 
ehoiees are influeneed by the so ealled “Peer Effeet”, “Neighborhood Effeet” or “Soeial Effeet”. 


In partieular, Broek and Durlauf use an approaeh similar to Mean Eied Games (MEG) [|26|, 
[ |27| and inspired by statistieal meehanics to study a static binary discrete choice model with a 
large number of agents which takes into account the effect of the agents’ interdependence on the 
individual choices. In their model, the individual choices are influenced by the mean of the other 
agents’ choices, while for an infinite size population, the impact of an isolated individual choice 
on this mean is negligible. The authors show that in an infinite size rational population, each 
agent can predict this mean as the result of a fixed point calculation, and makes a decentralized 
choice based upon its prediction. Moreover, multiple anticipated means may exist. Our analysis 
leads to similar insights for a dynamic non-cooperative multiple choice game including situations 
where the agents have limited information about the dynamics of other agents. 


II. Problem Statement and Contributions 

In this section, we formulate our problem, state our main contributions and provide an outline 
for the rest of the paper. 

A. Deterministic Case 

We consider a dynamic non-cooperative game involving N players with identical linear 
dynamics 

Xi = Axi + Bui yi e {1... ,N}, (1) 


where Xi G M” is the state of agent i and Ui G 
associated with an individual cost functional 


Ji { Ui , X , X °) = 


^11 

-\\Xi 
2 II 


x\ 




its control input. Player i, 1 < i < N, is 


where x{t) = Pj G M” (for j = 1 ,... , 1 ), q , r are positive constants and M 

is a large positive number. The running cost requires the agents to develop as little effort as 
possible while moving and to stay grouped around the mean of the population. Moreover, each 
agent should reach before the final time T one of the destinations pj,j = 1,..., / . Otherwise, 
it is strongly penalized by the terminal cost. Hence, the overall individual cost captures the 
problem faced by each agent of deciding between a finite set of alternatives, while trying to 
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remain close to the mean population trajectory. It is sometimes convenient to write the costs in a 
game theoretic form, i.e. Ji{ui, U-i), where U-i = (mi, ..., Mi_i, Wj+i,..., un)- We seek e-Nash 
strategies, i.e. such that an agent can benefit at most e through unilateral deviant behavior, with 


e going to zero as N goes to infinity p8| . We assume that each agent can observe only its own 
state and the initial states of the other agents. 

Definition 1: Consider N players, a set of strategy profiles S = SiX- ■ -xSn and for each player 
k, a payoff function ..., un), V(mi, ..., un) G S'. A strategy profile . .., G S' is 

called an e—Nash equilibrium with respect to the costs Jk if there exists an e > 0 such that for 
any fixed 1 < i < and for all Ui G Si, we have 




e. 


Inspired by the framework of MFG theory [I8|, [26|-[30| discussed in Section II-C below, 
we develop a class of decentralized strategies based on a fixed point requirement. Identification 
of the strategies requires only that an agent knows its own state and the initial states of the other 
agents. As we later show in the paper, when the number of agents N increases without bound, 
these fixed point based strategies achieve their meaning as e—Nash equilibria. 


B. Stochastic Case 


As N goes to infinity, it is also convenient to think of the initial states as realizations of random 
variables resulting from a common probability distribution function in a collection of independent 
experiments. Agent i, for 1 < i < A^, is then associated with the following adequately modified 
cost: 


Ji{ui,x,x^i) = E 


^11 -||2 , ^11 I 

l^\\Xi-x\\ + 


, M . 

at -\ -mm 

2 


\xi{T)-pj\\' 


X, 


(3) 


In this case, we establish that an agent only needs to know its own state and the common initial 
states probability distribution to construct one of the decentralized fixed point based strategies 
alluded to earlier. 


C. The MFG Approach and our Contributions 

The MFG approach is concerned with a class of dynamic non-cooperative games involving a 
large number of players where the individual strategies are considerably affected by the mass 
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behavior, while the influence of an isolated individual strategy on the group is negligible. Linear 


Quadratic Gaussian (LQG) MFG problems were developed in [26|, [281, [29|, while the general 


nonlinear stochastic framework was considered in [ |27| , pT|-p^. The MFG approach posits 
at the outset an infinite population to which one can ascribe a deterministic although initially 
unknown macroscopic behavior. Hence, one starts by assuming that the mean field contributed 
term x in the cost is given and equal to some x. The cost functions being now decoupled, 
each agent optimally tracks x. The resulting control laws are decentralized. This analysis of 


the tracking problem is presented in Section III By implementing the resulting decentralized 
strategies in the dynamics of the agents, a new candidate tracking path is obtained by computing 
the corresponding mean population trajectory. Indeed, and it is a fundamental argument in 
MFG analysis, asymptotically as the population grows, the posited tracked path is an acceptable 
candidate only if it is reproduced as the mean of the agents when they optimally respond to it. 
Thus, we look for candidate trajectories which are fixed points of the tracking path to tracking 
path map defined above. In Section |IV| these fixed points are studied for the deterministic case 
with a finite population, and an explicit expression is obtained by assuming that each agent 
knows the exact initial states of all other agents. The alternative probabilistic description of the 
agents’ initial states is explored in Section |Vj In Section VI we further generalize the problem 
formulation to include initial preferences towards the target destinations. Moreover, we consider 
that the agents have nonuniform dynamics and that each agent has limited information about 
the other agents dynamic parameters in the form of a statistical distribution over the matrices 


A and B. Section VII shows that the decentralized strategies developed when tracking the fixed 
point trajectories constitute e—Nash equilibria in all the cases considered above, with e going to 
zero as N goes to infinity. In Section |VIII[ we provide some numerical simulation results, while 
Section 1^ presents our conclusions. 

Although we rely on the MFG methodology in order to analyze the behavior of many agents 
choosing one of the available destinations, our model is not standard with respect to the LQG 
MFG literature. Specifically, our cost is non-convex and non-smooth in order to capture the 
combinatorial aspect of the discrete-decision making problem. Hence, the existence proofs for 
a fixed point rely here on topological fixed point theorems rather than a contraction argument 


as in [26|. One of the main contributions of this paper is also to show that in the case of a 


uniform population, the infinite dimensional MFG fixed point problem [27|, [311 has a finite 
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dimensional version that can be solved via Brouwer’s fixed point theorem [34|. For a nonuniform 
population, the existence of a fixed point path relies on an abstract fixed point theorem, namely 


Schauder’s fixed point theorem [34|. In both cases, to solve the MFG equation system, one 
needs to know the initial probability distribution of the players, whereas in the standard LQG 
MFG problems, it is sufficient to know the initial mean to anticipate the macroscopic behavior. 
Thus, in a nutshell, the theoretical tools needed to address this new formulation are thoroughly 
different. Further highlighting the differences between the two problems, the standard LQ MFG 
problem with stochastic dynamics is entirely tractable, whereas solving an extension to the 
current formulation with stochastic dynamics remains thus far beyond reach. 

Preliminary versions of our results appeared in the conference papers [[TJ, Q. Here we provide 
a unified discussion of our collective choice model for the deterministic and stochastic scenarios, 
as well as more extensive results. Many of the proofs were omitted from the conference papers 
due to space limitations and can be found here. The simulation section is also expanded with 
respect to Q, Q and provides additional insight on the role of the different parameters in the 
model. 


D. Notation 

The following notation is used throughout the paper. We denote by C(X,Y) the set of 
continuous functions from a normed vector space X to Y C with the standard supremum 
norm ||.||oo- We fix a generic probability space and denote by P(74) the probability 

of an event A, and by E(X) the expectation of a random variable X. The indicator function of 

O 

a subset X is denoted by lx and its interior by X. We denote by |X| the size of a finite set 
X. The transpose of a matrix M is denoted by M^. We denote by Ik the identity k x k matrix. 
The subscript i is used to index entities related to the agents, while the subscripts j and k are 
used to index entities related to the home destinations. We denote by [x]k the k-th component 
of a vector x. 
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III. Tracking Problem and Basins of attraction 
A. Tracking Problem 

Following the MFG approach, we assume the trajectory x{t) in (|^ and Q to be arbitrary for 


now and equal to x{t). The cost functions Q and Q can be written 


= min (wj,£,a:°), 


where 

Jij{Ui,X,X°) = 

Moreover, we have 


^\\Xi-x\\ +-\\Ui 


dt + ^\\xi{T) - pj\ 


(4) 


(5) 


inf JAui, X, x^) = min ( inf JiJui, x, x^) ) . 

Assuming a full (local) state feedback, the optimal control for Q is 


u, = Ui 


if Jij{u*^,x,x^) = mm ^Jik{u*^,x,x^), 


where u*^ is the optimal solution of the simple linear quadratic tracking problem with cost 


function Jik. We recall the optimal control laws [35| 

with the corresponding optimal (simple) costs 

^ „ 0 nT 




= -(7)^r(o)7 + /?7o)^T:« + 4(o), 

where F, f3k and 5k are respectively matrix-, vector-, and real-valued functions satisfying the 
following backward propagating differential equations: 


r - -tbb'^t + fa + A^r + qin = o 


Pk = _ A^ J + gx 

5k = ^{(dkf'BB'^(3k - ^qx'^x, 


(6a) 

(6b) 

(6c) 


with the final conditions 


r(T) = M4, (3k{T) = -Mpk, 5k{T) =-Mplpk. 
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We define the basins of attraetion 


Dj{x) = jx e M"" sueh that (/3j(0) — f3k{0))'^x < 5^(0) — 5j(0), V/c = 1,...,(7) 


for j = 1,If an agent i is initially in Dj{x), then the smallest optimal (simple) eost is J*j, 
and player i goes towards the eorresponding destination point pj. 

Assumption 1: Conventionally, we assume that if G ^r some ji < ■ ■ ■ < jk, 

then the player i goes towards pj^. Under Assumptions [^and[^ this eonvention does not affeet 
the analysis in case of random initial conditions. 

We summarize the above analysis in the following lemma. 

Lemma 1: Under Assumption the tracking problem Q has a unique optimal control law 


= — B^{T{t)xi + (3j{t)) 


if e Dj{x), 


( 8 ) 


where T, (3j, 5j are the unique solutions of (6a)-(6c). 

The optimal control laws ([^ depend on the tracked path x{t) and the local state Xi. As 
mentioned above, each agent should reach one of the predefined destinations. We show in the 
next lemma that for any horizon length T, M can be made large enough that each agent reaches 
an arbitrarily small neighborhood of some destination point by applying the control law Q. The 
result is proved for tracked paths x{t) that are uniformly bounded with respect to M, a property 
that is shown to hold later in Lemmafor the desired tracked paths (fixed point tracked paths). 


Lemma 2: Suppose that the pair (A, B) is controllable and for each M > 0, the agents are 
optimally tracking a path XM{t)- We suppose that the family XM{t) is uniformly bounded with 
respect to M for the norm ^ ||.|pdtj Then, for any e > 0, there exists Mq > 0 such that 
for all M > Mo, each agent is at time T in a ball of radius e and centered at one of the p/s, 
for j = 1 ,..., 1. 

Proof: See Appendix ■ 

Given any continuous path x{f), there exist I basins of attraction where all the agents initially 
in Dj { x ) prefer going towards p^, j = 1, ..., / . Therefore, the mean of the population is highly 
dependent on the structure of Dj { x ), j = 1..., 1. In the next paragraph, we study the properties 
of these basins in more detail. 
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B. Basins of Attraction 


We start by giving an explicit solution of (6b) and (6c). Let n(t) = ^V{f)BB'^ — and 
for 7] G M, be the unique solution of 


dt 




$(77,77) = In, 


and 




(9) 


( 10 ) 


Two main properties of the state transition matrix $ are used in this paper, namely the matrix 
*^*(^ 1 ,^ 2 ) has an inverse $( 772 , 771 ) and the state transition matrix $( 771 , 772 ) of — 11 ^ is equal to 
$( 772 , 771 )^. For more details about the properties of the state transition matrix, one can refer to 
1. We have 

/3k{t) =-M^{t,T)pk + q [ ^{t,a)x{a)da 


6k{t) = ^Mplpk - I y x{a)'^x{a) da 
+ Tj-Pfc / d/{p,T,r],T)dppk - pI 


2r 

2r 


$( 77 , T, 77 , a)x{a) dadp 


It Jt 


r -7 pri pri 


x{a)'^dt{ri, a, rj, t)x{t) drdadp. 


It Jt 


By replacing (|TT]) in the expression of Dj{x), (|^ can be written 


DAx) = < X e 


such that iSjkX < 6jk + ajk{x), Wk = 1,... ,l 


where 


(3jk = M^{0,T){pk-pj) 
Sjk = ^Mplpk - ^Mpjpj 




+ -AirPk I T, 77 , T) d77Pfc - —Pj 


Uljk (T) 


2r 

Mq 


2r 


'^{V,T,V,T) dppj 


{Pj - Pk? 


0 prj 



$( 77 , T, 77 , a)x(a) dadp. 


T JT 


( 11 ) 


( 12 ) 


(13) 
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IV. Fixed Point - Deterministic Case 


Having presented the solution of the general tracking problem, we now seek a continuous 
path x{t) that is sustainable, in the sense that it can be replicated by the mean of the agents 
under their optimal tracking control laws. We start by analyzing the finite size population where 


the initial state of each agent is known to all the agents. We start our search for the desired path 
x{t) by computing the mean x{t) when tracking any continuous path x{t). The dynamics of the 
mean when tracking x G C([0,T],M’^) satisfies 



where x(0) = Px{x) = Yl]=i the number of agents initially 


in Dj{x), which therefore pick pj as a destination. We obtain (14) by substituting (|TT]) in ([^ and 



Thus, the mean of the population x when tracking any continuous path x is the image of x by 
a composite map G = G2 o Gi, where 


Gi : C'([0,T],M") ^ C'([0,T],M’^) x N' 



G 2 : C([0,T],M”) X N' ^ C([0,T],M’*) 




is the unique solution of (14) in which Xj{x) is equal to an 


and X 


arbitrary Xj, j = 1 ,..., /. 

The desired path describing the mean trajectory is a fixed point of G. In the following, we con¬ 
struct a one to one map between the fixed points of G and the fixed points of di finite dimensional 
operator F describing the way the population splits between the destination points. We start by 
showing that the fixed points of G have a special form. For any A = (Ai,..., A/) G {0,..., NY, 
we define a new map from (^([OjT],]^") to (7([0,T],M”), where Tx{x) = G 2 {x,X). If x is 
a fixed point of G and Aq = A(i;), then f is a fixed point of In the following lemma, we 
show that for any A, Tx has a unique fixed point yx, and we give an explicit form for yx- 

Lemma 3: For all A = (Ai,..., Xi) G {0,..., Ny, Tx has a unique fixed point equal to 


yx = Riit)xo + R 2 it)px, 


(15) 


Januaiy 26, 2016 


DRAFT 






11 


where 


Ri{t) — $p(t, 0) 

M R 

R^{t) = — / ^p{t,a)BB^^p{T,afda, 
r Jo 


(16) 


and P and ^p(t,r]) are the unique solutions of 


P = -PA - A^P + -PBB^P, P(T) = Min 
r 

$p(t, r]) = -{A- -BB^Pf^p{t, v), <hp(r 7 , r]) = R 


(17) 


Moreover, if (A, B) is eontrollable, then the paths y\ are uniformly bounded with respeet to M 
for the norm ( fj' ||.||^dt 


Proof: See Appendix ■ 

The fixed point path ( [T5] ) is the optimal state of the LQR problem Q, where g = 0 and the 
final destination point is px. 

Hitherto, we know that the fixed points of G are of the form ( [T5] ). To narrow the seareh 
further, we derive a neeessary eondition on the veetor A so that the eorresponding yx is a fixed 


point of G. We start by replaeing the new expression of the fixed points (15) in the expressions 
of the basins of attraetion, whieh then have the following form: 


= Dj{Ri{t)xo + R2{t)p\) = {x e < 6jk + OjkXo + RkPx Vfc = 1,..., / 


where 


Gjk — 


-ipj-pl) 


fO pr) 


m{p,T,p,a)Ri{a) dadp 


Mq rp rp 

ijk = —{Pj -Pk) 


fO prj 


^{p,T,r],a)R2{(j) dadp. 


(18) 


(19) 


Following the diseussion above and Lemma we ean elaim that if x is a fixed point of G, then 
X is of the form ([T5]), where 


A = A(x) = (l{x°|x° e L>i(x)}|,...,|{x°|xi e A(x)}|) 
= (|{x°|x- e H^}1 ..., |{x°|x- e A"}|) ^ F(A). 


( 20 ) 
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Thus, we proved that if a; is a fixed point of G, then x is of the form ([T5|), where A is a fixed 


point of the finite dimensional operator F defined in (20). To prove the eonverse, we eonsider 
a fixed point A of F and the path x = Ri{t)xQ + i? 2 (f)pA- We have 

A = F{X) = (|{i»|4 e 6 

= (l{4k“ e l{4k“ e A(i)}|) = A(f), 


where the third equality is a eonsequenee of the form of x. The path x is the unique fixed point 
of Tx- But X = Tx{x) = Tx(x)ix) = G{x). Therefore, f is a fixed point of G. We summarize the 
above diseussion in the following theorem. 

Theorem 4: The path f is a fixed point of G if and only if it has the form ( [T5| ), where A is a 
fixed point of F. 

Without loss of generality, we ean index in the binary ehoiee ease (/ = 2) the agents going 
towards pi by numbers lower than those given to the agents going towards p 2 as follows: 

( 21 ) 


Then, the neeessary and suffieient eondition for the existenee of the desired path reduees to a 
simple inequality as shown in the following theorem. 

Theorem 5: For I = 2, the following statements hold: 

1) a; is a fixed point of G if and only if there exists a seperating a in {0, ■■■, N} sueh that: 
For a different from 0 and N, 

ol 

- ')l 2 - ^ 12^0 - i\2V2 < ^^ 12 (Pl - V2) < “ ^12 “ “ ^ 12 ^ 2 , (22) 


For a = 0, 0 < {l 4 i 2 Yx^ - 812 - 9 i 2 Xq - ^ 12 ^ 2 , (23) 

For a = N, {l 3 i 2 Yx% - 812 - 612 X 0 - ^ 12^2 < 0. (24) 


2 ) 

3) 


In this ease, a is the number of agents that go towards pi. 


For ii2{pi — P2) > 0, there exists a in {0, satisfying (22), (23) or (24). 


For ^i 2 (pi — P2) < 0, there exists at most one a in {0,..., A^} satisfying (22), (23) or (24). 
Moreover, there exist some initial distributions for whieh no sueh a exists. 

Proof: See Appendix ■ 
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Remark 1 : For the scalar case {n = 1), ^12(^1 — P2) is always non-negative. In fact, in this 
case, <I) and are real exponential functions. This implies that .^ 12(^1 — P2) > 0. 

Theorem shows that computing the anticipated macroscopic behaviors (fixed points of G) 
is equivalent to computing all the fixed points A’s of F for which it is necessary to assume that 
each agent knows the exact initial states of all the other agents. Thus, for each A G {0,..., 
such that I'^X/N = 1, the agents must count the number of initial positions rjj inside each region 
Hj, j = 1,. .., /. If rjj = Xj, for j = 1, ..., I, then A is a fixed point of F. The map F may 
have multiple fixed points. Hence, an a priori agreement on how to choose A should exist. For 
example, although non-cooperative, the agents may anticipate that their majority will look for 
the most socially favorable Nash equilibrium if many exist and N is large. This A corresponds 
to minimizing the total cost ^ + 

/3fc(0)^x° -f 5fc(0)|, which is also computable by just knowing the exact initial conditions of all 
the agents. Once the agents agree on a A, they start tracking the corresponding fixed point defined 
by ( [T5] ). The fixed point vector A describes the way the population splits between the destination 
points. In fact, Xj, j = 1,...,/, is the number of agents that go towards pj. When N is large, 
this algorithm is costly in terms of number of counting and verification operations. In the next 
section, we consider the limiting case of a large population with random initial conditions. 


V. Fixed Point - Stochastic Case 


In this section, we assume that the agents’ initial conditions are random and i.i.d. on some 
probability space with distribution Pq on M”. In this case, we show that for a large 

population, it is enough to know Pq to anticipate the macroscopic behavior. For a continuous 
path X and for all ce G H, we denote by AjM{u) the number of x°(a;) in Dj{x), by XNiuj) the 
mean of the population when tracking x, and by x^a the limit with probability one of tat as 
goes to infinity. We deduce from ([T4|) that for all 00 in fl. 




N 


i=i 


N 


\h(cT, f, a, T)pj da 


Q 

r 


'k(cr, t, a, t)x{t) drdcr, 


where \k is defined in (10). By the strong Law of large numbers, 




and 




N 


converge with probability one respectively to Po(-Dj(a;)) = P(a;° G Dj{x)] and 
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Ea:° = /io, as iV goes to infinity. Hence, 


Xoc = ^{0,t)^fio + '^Po{Dj{x)) — / ^((T,t,cr,T)pjdcr 
,=i ^ Jo 


q 

r 


a,T)x{T) drda. (25) 


Equation ( [25| ) defines an operator that maps the tracked path x to the mean Xoo- This operator 
and its fixed points, if any, depend only on the initial statistical distribution of the agents. The 


limiting equation (25) also corresponds to the following stochastic problem. Assume that the 
only public information is the initial statistical distribution. As in the deterministic case, we start 
our search for a fixed point path by replacing x in ([^ by a continuous path x. By Lemma 
there exist I regions Dj(x) such that the agents initially in Dj(x) select the control law ([^ when 
tracking x. By substituting ( [TT] ) in ([^ and the resulting control law in ([^, we show that the 
mean trajectory E(xi) of a generic agent is equal to Gs{x). 

The next theorem establishes the existence of a fixed point of Gg- We define the set A; = 
{(Ai,..., A/) e [0,1]^| X]j=i the map Fg from A/ into itself such that 


Fs(Ai, . . . , A;) — 


p 


(pTjx'^i < (5ii + OijiiQ + iijPx, Vj = 1,..., /) 
< 5ij + 9ijHo + ^ijpx, Vj = 1,..., 


where p\ = Yl\=i^kPk- The quantities (dkj and 5kj are defined in (13), and 6 kj and ^kj are 
defined in ( [T8] ) and ( [79] ). 

Assumption 2: We assume that Pq is such that the Po-me^sure of hyperplanes is zero. 
Theorem 6: Under Assumption the following statements hold: 

(i) X is a fixed point of Gg if and only if there exists A = (Ai,..., A;) in A; such that 


Fs{X) = A, 


(26) 


for x(t) = Pi(f)/io + R 2 {t)px. 

(ii) Fg has at least one fixed point (equivalently Gg has at least one fixed point). 

(iii) For / = 2, if ^ 12(^1 — P2) < 0, then Gg has a unique fixed point. 

Proof: See Appendix ■ 
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The finite dimensional operators F and Fg defined respeetively in the deterministie and stoehastie 
eases have similar structures. In fact, in the deterministic case, if the sequence of initial 

conditions is interpreted as a random variable on some probability space (fl, F, P) with distri¬ 
bution Po{A) = (Borel) measurable sets A, then F{X) = NFs{X/N). 

In Theorem (i) shows that computing the anticipated macroscopic behaviors is equivalent to 


computing all the vectors A satisfying (26) under the corresponding constraint on x. To compute 


a A satisfying (26), each agent is assumed to know the initial statistical distribution of the agents. 
As in the deterministic case, multiple A’s may exist. Hence, an a priori agreement on how to 
choose A should exist. In that respect, the agents may implicitly assume that collectively they 
will opt for the A (assuming it is unique!) that minimizes the total expected population cost 

EJi(u*{xi,x),x),x^) = E^min^|^(a;-)^r(0)a;° + /?fc(0)^a;° + 4(0)}, 

which can be evaluated if the agents know the initial statistical distribution of the population. 


A. Computation of The Fixed Points 

The map Fg is not necessarily a contraction. Hence, it is sometimes impossible to compute 
its fixed points by the simple iterative method Afc+i — Fg{Xk). 

1) Binary Choice Case: We give two simple methods to compute a fixed point of Fg in the 
binary choice case. The first method is applicable if ^ 12(^1 — P2) > 0. We define in [0,1] a 
sequence ak such that is an arbitrary number in [ 0 , 1 ] and 


— {cik+ii 1 — ttfc+i) — Fg(ak, 1 — ak) — Fg(Xk). 


(27) 


Given that 62(^1 - P2) > 0, Fg{t, I - t) 


increases with t. We show by induction that ak is 


monotone. But ak G [0,1], therefore, ak converges to some limit a. By the continuity of Fg, 


{a, 1 — a) satisfies (26). Since in this case Fg may have multiple fixed points, the A = (a, 1 — a) 
obtained using this approach depends on the initial value Aq = (oq, 1 — oq). If we define 


Xk = Rlit)lJ,o + R2{f)px^,. 


(28) 


This sequence converges to a fixed point of Gg. The second method is applicable if ^ 12 (^ 1 —P 2 ) < 
Fs{X, 1 — A)]^ — A decreases with A. Hence, one can compute the unique zero 


0. In this case 


of this function by the bisection method. 
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2) General Case: In general {I > 2), Fg is a vector of probabilities of some regions delimited 
by hyperplanes. Although a fixed point could be computed using Newton’s method, this is 
computationally expensive as it requires the values of the inverse of the Jacobian matrix at the 
root estimates. Alternatively, one can compute a fixed point of Fg using a quasi Newton method 
such as Broyden’s method [37| ( see Section [Vni ). Using this method, the inverse of the Jacobian 
can be estimated recursively provided that Fg is continuously differentiable; this will be the case 
if the initial probability distribution has a continuous probability density function. 


B. Gaussian Binary Choice Case 

We showed in Theorem that for the binary choice case (/ = 2), if ^ 12(^1 — P 2 ) < 0, then 
Gg has a unique fixed point. We now prove that for the binary choice case and Gaussian initial 
distribution irrespective of the sign of ^uiPi — P 2 ), Gg has a unique fixed point provided that 
the initial spread of the agents is “sufficient”. For any nxn matrix Sq such that (/5i2)^So/3i2 < 
{^Mpi - P2)Y/271, we define 

a(So) = ^12 + 62P2 - V‘^Wi‘ 2 V^o/ 3 i 2 ^^og^i 2 {pi -P2) - ^ Iog 27 r(/?i 2 )^llo/ 3 l 2 

^(^o) = <^12 + il 2 Pl + s/2{(5l2)'^^ol3l2\j^Og^i2{pi — P 2 ) “ ^ 27r(/3i2)^So/3l2 

5(So) = {/io e (/??; - 0i2)/ro e (a(So), 5(So))}. 


Theorem 7: Gg has a unique fixed point if at least one of the following conditions is satisfied: 

1) /5f2So/3l2 > • 

2) /ro i ^(So). 

Proof: See Appendix ■ 

Theorem states that in the Gaussian binary choice case, if the initial distribution of the 
agents has enough spread, then the agents will anticipate the collective behavior in a unique 
way. On the other hand, if the uncertainty in their initial positions is low enough and the mean 
of population is inside the region S'(So) (a region delimited by two parallel hyperplanes), then 
the agents can anticipate the collective behavior in multiple ways. 
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VI. Nonuniform Population with Initial Preferences 


Hitherto, the agents’ initial affinities towards different potential targets are dictated only by 
their initial positions in space. In this section, the model is further generalized by considering 
that in addition to their initial positions, the agents are affected by their a priori opinion. 


When modeling smoking decision in schools for example [381, this could represent a teenager’s 
tendency towards “Smoking” or “Not Smoking”, which is the result of some endogenous factors 
such as parental pressure, financial condition, health, etc. When modeling elections, this would 
reflect personal preferences that transcend party lines. Moreover, we assume in this section that 
the agents have nonuniform dynamics. 

We consider N agents with nonuniform dynamics 


Xi = AiXi + BiUi 




(29) 


with random initial states as in Section |Vj Player i, i = 1,..., iV, is associated with the following 
individual cost: 


Ji{ui,x,x°) = E 




x\ 


+ 


dt + min 




X, 


(30) 


As N tends to infinity, it is convenient to represent the limiting sequence of = 

((Aj, Bi, Mil, • • ■, by a random vector 9. We assume that 6* is in a compact set 0. 

Let us denote the empirical measure of the sequence 9i as {A) = ^{9ieA} for 

(Borel) measurable sets A. We assume that has a weak limit Pq, that is for all 0 continuous, 
limAr^.oo/e 0(a^)dPj^(x) = jQ(j){x)dPe{x). For further discussions about this assumption, one 


can refer to [39|. We assume that the initial states x^ and 9 are independent. 


In the costs ( |3()| ), a small relative to Mik, k ^ j, reflects an a priori affinity of agent i 
towards the destination pj. We assume that an agent i knows its initial position x?, its parameters 
9i, as well as the distributions Pq and Pq. We develop the following analysis for a generic agent 
with an initial position and parameters 9. Assuming an infinite size population, we start by 
tracking x{t), a posited deterministic although initially unknown continuous path. We can then 
show that, under the convention in Assumption this tracking problem is associated with a 
unique optimal control law 




if e DAx), 


(31) 
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where F®, 6 j are the unique solutions of 


fj - + rfA® + + g/n = 0 


$^=( - {AyAp^^+qx 


■^e rte/ 


e\T 


1 ' q 9\T nO/ r}e\T nO ^ 


6^^ = -{yyB%Byy--qx^x, 


(32a) 

(32b) 

(32e) 


with the final eonditions r^(T) = M^n, AjiT) = —M^j^SyT) = ^MjpJpj. The definition 
of the basins of attraetion beeomes 


DAx) = lxe 


sueh that x^F^^a; + x'^y^x) + S^^^x) < 0, V/c = 1,..., Z }>, (33) 


jk 


jk\ 


where 


r% = r»(o) - r»(o) 

D 0 


(34) 


- Atio) 

6%{x) = 6%0)-6t{0). 

In this ease, the solutions of the Rieeati equations ( 32a[ ) depend on both the initial preferenee 
veetor and the destination points. Henee, the basins of attraetion are now regions delimited 
by quadrie surfaees in instead of hyperplanes. This faet eomplieates the strueture of the 
operator that maps the traeked path to the mean. The existenee proof for a fixed point relies 
now on an abstraet Banaeh spaee version of Brouwer’s fixed point theorem, namely Sehauder’s 
fixed point theorem [34|. We define 

where Ft®(f) = yy)B^{B^Y ~ {Ay, and y is defined as in (|^, where FI is replaeed by 
n®. The state trajeetory of the generie agent is then 


X 


oe 


(f) = ^1^9(£)(x°)|<I)^®(0,f)^x° +^ 

,=i ^ ^ r Jo 


1 

r 


4/® {a, t, a, t)x{t) drdcr 


Assumption 3: We assume that E||x°|f < oo. 


The funetions defined by (32a), (32b) and (32e) are eontinuous with respeet to 9 whieh belongs 
to a eompaet set. Moreover, 6 and x° are assumed to be independent. Thus, under Assumption 
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the mean of the infinite size population ean be eomputed using Fubini-Tonelli’s theorem |40| 
as follows: 


x{t) = E(a;°®(f)) = = Ex\t) = ^ / / 

^ ^ -I Je ^ '■ 


+ -^y '^^j{a,t,a,T)pjda - ^ J j 4^®(cr, f, a, r)£(r) drdcr| dPod-Pe, (35) 

where x^(t) = E(a:°® (f)|6*). Equation (p^ defines an operator Gp from the Banaeh spaee 
(C([0,T],M"'), ||.||oo) into itself whieh maps the infinite population traeked path x to the eorre- 
sponding mean x, itself eonsidered as another potential traeked path. 

In the next theorem, we show that Gp has a fixed point. We define 

fci = E||x°|| X ( V max ||$?(0,E 
\ ^ (0,t)eex[o,T]" ' 

\i=i 


i=i 

(*t pa 


/c2 = / max 

{e,t)&ex[o,T] 

i=i 


m: 


e rt 




(36) 


i 




max \\wAa,t,a,T)\\. 
r ^ (0,t,CT,T)e0x[o,r]3" 
i=i 


Sinee 0 and [0,T] are eompaet and is eontinuous with respeet to time and parameter 9, 
then ki, k 2 and k^ are well defined. 


Assumption 4: We assume that A/max(fci + k 2 , k^jT < 7r/2. 

Noting that the left hand side of the inequality tends to zero as T goes to zero, Assumption 
1^ ean be satisfied for short time horizon T for example. 

Assumption 5: We assume that Pq is sueh that the Po-measure of quadrie surfaees is zero. 
Theorem 8: Under Assumptions and Gp has a fixed point. 

Proof: See Appendix ■ 


VII. Nash Equilibrium 

In the three eases above, deterministie, stoehastie and stoehastie with initial preferenees, we 
defined three maps G, Gg and Gp respeetively. Depending on the strueture of the game, eaeh 
player ean antieipate the maeroseopie behavior of the limiting population by eomputing a fixed 
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point X of G, Gs or Gp, and compute its best response u*{xi,x) to x as defined in ( [3T| ). 
When eonsidering the finite population, the next theorem establishes the importanee of sueh 
deeentralized strategies in that they lead to an e-Nash equilibrium with respeet to the costs 
Q and ( [^ . This equilibrium makes the group’s behavior robust in the faee of potential selfish 
behaviors as unilateral deviations from the assoeiated eontrol polieies are guaranteed to yield 
negligible eost reductions as N increases sufficiently. 

Theorem 9: Under Assumption]^ the deeentralized strategies = defined in ([^ 

and pT] ) for a fixed point path x, constitute an CAr-Nash equilibrium with respect to the costs 
Ji{ui,u-i), where cat goes to zero as N inereases to infinity. 

Proof: See Appendix ■ 


VIII. Simulation Results 

To illustrate the eollective decision-making mechanism, we eonsider a group of agents moving 


m 


aeeording to the dynamies 

0 


A = 


0.02 -0.3 


B = 


0 

0.3 


towards the potential destination points pi = (—39.3, —10), p 2 = (—27, 9.5) or pa = (0,40). We 

1 T 


-10 0 


,5/2 .We 


draw N = 600 initial eonditions from the Gaussian distribution Pq := ff 
simulate two cases. In the first one, eaeh agent knows the exaet initial states of the other agents 
and antieipates the mean of the population accordingly. Following the counting and verification 
operations deseribed at the end of Section we find that F has multiple fixed points, for 
example, A = (564,11,25). By implementing the eontrol laws eorresponding to this partieular 
A, 564 agents go towards pi, 11 towards p 2 and the rest towards ps (see Fig0. Moreover, the 
aetual average replieates the anticipated mean as shown in this figure. In the seeond case, the 
agents know only the initial distribution Pq of the agents. Then, Broyden’s method eonverges to 


A = (0.9162,0.0258,0.058) satisfying (26). Accordingly, 91.62% of the agents go towards pi, 
2.58% towards p 2 and the rest towards pa (see Fig|^. The actual average and the antieipated 
mean are approximately the same. 

To illustrate the soeial effeet on the individual ehoiees (see Fig. [^, we eonsider the same 


initial conditions. Without social effect (q = 0), (0,0.25,0.75) satisfies (26). In this case, the 
majority goes towards ps. As the soeial effeet inereases to q = A, some of the agents that went 
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towards pi or p 2 in the absence of a social effect change their decisions and follow the majority 
towards p^ (see blue balls in Fig. |^. In this case, (0, 0.16, 0.84) satisfies (261. If the social impact 
increases more to g = 6, then a consensus to follow the majority occurs. 

To illustrate the impact of the individual efforts on the behavior of the population (see Fig. 
1^, we start with the case where control effort is inexpensive (r = 3) relative to the social effect 
(q = 14). In this case, each agent prefers following the majority. A consensus to go towards 
P 3 occurs. As the effort coefficient increases (r = 30), some of the agents prefer going to a 
less expensive destination (pi) than following the majority. By increasing more the penalty on 
the effort (r = 60), a third party appears. This subgroup goes towards p 2 . Moreover, when r 
decreases, the agents reach smaller neighborhoods of the destination points. 

To illustrate the Gaussian Binary Choice Case, we consider a population of = 500 agents 
initially drawn from the normal distribution A/^(/io, I 5 / 2 ) and moving in according to the 
dynamics A = B = I 2 towards the destination points pi = (—20,0) or p 2 = (20,0). For this 
covariance matrix Sq = I 5 / 2 , S'(So) is the region delimited by the vertical lines x = —15 
and X = 15. If po = (18 5), i.e. outside S'(So), only one e—Nash equilibrium corresponding 
to A = (0,1) exists. If /iq = (0.5 5), i.e. inside S'(So), three e-Nash equilibria exist. The first 
corresponds to Ai = (0.89,0.11) (Fig. |^, the second to A 2 = (0.4, 0.6) (Fig. |^, and the third 
to As = (0,1) (Fig.g. 


IX. Conclusion 

We consider in this paper a dynamic collective choice model where a large number of agents 
are choosing between multiple destination points while taking into account the social effect as 
represented by the mean of the population. The analysis is carried using the MFC methodology. 
We show that under this social effect, the population may split between the destination points in 
different ways. For a uniform population, we show that there exists a one to one map between the 
fixed point behaviors (anticipated behaviors) and the fixed points of an operator defined on W. 
The latter describe the way the agents split between the I destination points. Finally, we prove that 
the decentralized strategies developed while tracking the anticipated behaviors are approximate 
Nash equilibria. For future work, it is of interest to analyze a model where stochasticity is 
extended to the players’ dynamics as well. In that case, the optimal choices (feedback strategies) 
are adapted to the underlying filtration and change along the path. This is in contrast to the 
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Fig. 1. Collective choice - deterministic initial conditions - A = (564,11, 25) 




Fig. 2. Collective choice - random initial conditions - A = (0.9162,0.0258,0.058) 
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q = 0 (No Social Effect) 




40r 
30 - 
20 - 
10 - 
0 - 

- 10 ^-^^- L 

-40 -35 -30 -25 


q = 6 





0 5 


Fig. 3. Influence of the social effect q 


current formulation where the agents ean ehoose without loss of optimality their destination 
before they start moving. Moreover, we would like to extend the eurrent formulation to eertain 
nonlinear models, where the basins of attraetion are delimited by more eomplex manifolds, and 
the fixed-point eomputations would require numerieal methods for baekward-forward systems 


of partial differential equations [411. 


Appendix A 

A. Proof of Lemma 

In this proof, the subseript M indieates the dependenee on the final eost’s eoeffieient M. For 
any M > 0, the agents are optimally traeking a path xm- The agent’s i optimal state is denoted 
by We have 

y ^min^ (\\x*m{T) - x°), 

where is the eost define by ([^ with the final eost’s eoeffieient equal M. It 

suffiees to find an upper bound for xm, x^) whieh is uniformly bounded with M. Sinee 
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r = 3 


-4D 


r = 30 




‘-"O 


r = 60 



Fig. 4. Influence of r 


2D 



-25 -20 -15 -10 -S 0 5 10 15 20 25 

Coordinate x 


Fig. 5. Gaussian binary choice case - Multiple equilibria - A = (0.89,0.11) 

{A, B) is controllable, then there exists for each agent i a eontinuous control law Uxq p_^{t) on 
[0,T] which transfers this agent from the state to pi in a finite time T. By optimality, we 
have 

1 ^ JiM ( , Xj j . 
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-25 -20 -15 -10 -5 0 5 10 15 20 25 

Coordinate x 


Fig. 6. Gaussian binary choice case - Multiple equilibria - A = (0.4, 0.6) 


20 



-25 -20 -15 -10 -5 0 5 10 15 20 25 

Coordinate x 


Fig. 7. Gaussian binary choice case - Multiple equilibria - A = (0,1) 


But, 




iM I ^M 7 


q T* 

2 ^m|| 


dt, 


which is uniformly bounded with M, since xm is uniformly bounded with M. Thus, for all 
e > 0, there exists an Mq > 0 such that for all M > Mq, 

^min^ {\\x*m{T) - I < e. 


B. Proof of Lemma 

Lets consider y a fixed point of Tx. We define 

n(t) = T{t)y{t) + q j <h(f, a)y{a) - T)px. 

One can easily check that {y, n) satisfies 

1 ^ 

y = Ay - BB n 

r 

h = —A^n 


(37) 
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y and n are respectively the optimal state and co-state of the following LQR problem: 


/■-' r M 

min / -||M||^dt-I-—||a:(T) — paIT 
'o 


(38) 


Subject io X = Ax + Bu 


x(0) = Xq. 


Therefore, n has the representation n{t) = P{t)y{t) + g{t), where P is the unique solution of 
the Riccati equation ([T7|) and g satisfies 


g = -{A--BB'^P{t)fg 


g{T) = -Mpx. 


By solving g and implementing its expression in n = Py + g, and by implementing the new 
expression of n in the dynamics of y, one can show that y{t) = Ri{t)xo+R 2 {t)px. Conversely, let 


n,y) the unique solution of (371. We define m(t) = T(t)y(t)+q <h(f, a)y{a)da—M^(t, T)px. 


One can easily check that {m — n) = {^TBB^ — AA'){m — n), with m{T) = n{T). Therefore, 
m = n. Hence, p is a fixed point of ( fT?] ). We now prove the uniform boundedness of the fixed 
point paths yx with respect to M. The paths yx are the optimal states of the control problem 


(38). Since {A,B) is controllable, one can show that the corresponding optimal control law ux 


satisfies 


I < I 


Mt, 


where mq is a continuous control law that transfers the state y from y{R) to px- uq is independent 
of M. We have 

yx{t) = exp{At)xo + / exp{A{t — a))Bux{<j)da. 

Jo 

Therefore, 

^ \\yxfdt<K, + K2j^ \\uxfdt + Ks(^J^ ||MA||Mt 

for some positive constants which are independent of M. Hence, yx is uniformly 

bounded with M. 


C. Proof of Theorem 

The first point follows from Theorem]^ and (21). For 2) and 3), we define 

■(/^12^a — 5 i 2 — O12XQ — ^12^2)- 


N 




62(^1 -P2) 
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We start by proving 2). Suppose that there does not exist any a in {0, satisfying (22), 


or (24). Zero does not satisfy (23), henee aAr(l) < 0 < 1. One does not satisfy (22) and 
aAr(l) < 1, henee aAr(2) < 1. By induetion, we have aAr(iV) < N — 1. Therefore, N satisfies 


|). Thus, by eontradietion, there exists a in {0, satisfying (22), (23) or (24). We now 

prove the third point. Suppose that there exist multiple a’s satisfying ([22]), (|23|) or ([24]). Let ao be 


the least of these a’s. If oq < N, then in view of ^ 12(^1 — P2) < 0, aAr(ao + 1) < oq < aAr(ao)- 
a]\f{j) is deereasing. Henee, for all a > ao, ajv(a) < aAr(ao +1) < Oq < «• Therefore, Oq is the 


unique a satisfying (22), (23) or (24). If .^i 2 (pi —P 2 ) < 0, then the initial distribution for whieh 


aAr(a) is in (0,1) for all a in {0,..., iV} does not have any a in {0,..., iV} satisfying ( [^ , ( [^ 
or 


Appendix B 


A. Proof of Theorem 


We start by proving (i). Let a; be a fixed point of Gg and Xj = Po^Dj^x)). By replaeing 
the probabilities in the expression of Gg by Xj, j = 1,...,/, we get f = Gg{x) = Tx{x), 
where A = (Ai, .. . ,Xi) and T\ are as defined in ( [T^ . Henee, f is a fixed point of T\. By 
Lemma x{t) = + R2{t)p\. By replaeing this expression of x in Dj{x), we get 

A = Fs{X). Conversely, eonsider A = (Ai,..., A;) in Ai sueh that A = Fg{X) and let x{t) = 
Riit)po + i? 2 (f)PA- The path x is the unique fixed point of Tx and 

(^Po{D,{x)),PoiDiix))^ = Fg{X) = A. 


Henee, x = Tx{x) = Gg{x). We now prove the seeond point. Noting that the set A; is eonvex 
and compaet in M}, we just need to show that Fg is eontinuous. Then, Brouwer’s fixed point 


theorem [34| ensures the existenee of a fixed point. Let A^ be a sequenee in eonverging to 
A. Let 


Dkr = |x e M" sueh that {/3kj)'^x < 5kj + OkjPo + ikjPK, Vj = 1 ,..., / | 
Dfc = |x e M*" sueh that {(^kjf x < 6kj + 9kjPo + ^kjP\, Vj = 1,..., / 


We have 





(x)-1d,(x) dP„(a:). 
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But, Dkr and are regions delimited by hyperplanes. Hence, under Assumption 


-lDk{x) dPo(a;) = / 1 ° (a;)-lo {x) dPo^x). 


Dk 


Dk 


But, 


lo (a;)-lo (x) 

Dkr Dk 


< 2 and converges to zero for all x in M”. Thus, by Lebesgue dominated 
convergence theorem p0| , the integral of this function converges to zero. This proves that Fg is 
continuous. Finally, we prove (iii). For I = 2, the fixed points of Fg are of the form (a, 1 — a). 
The set of the fixed points of Fg is compact. Thus, the set of the first components of these fixed 
points is compact. Let cto be the minimum of those first components. Consider a > Uq. Hence, 


- <5i 2 - 6'i2/io - ^12P2 < - 


c {{/3i 2)^X^ - 8x2 - 6 'i 2 /iO - 62 P 2 < Oloil2{pi 


which implies 


Fg{a, 1 - a) < Fg{aQ, 1 - ao) 


— ClQ < ol. 


J 1 L J 1 

Thus, (ao, 1 — ao) is the unique fixed point of Fg, and x{t) = Pi(f)/ 2 o + R 2 {t)P(ao,i-ao) is the 
unique fixed point of Gg. 

B. Proof of Theorem 

We show in Theorem that the fixed points of Gg can be one to one mapped to the fixed 
points of Fg. The initial states x° are distributed according to a Gaussian distribution Sq). 

Therefore, are distributed according to the normal distribution Af(^(3'[2To, ■ Thus, 

one can analyze the dependence of [Fs(a, 1 — a)]^ — a on a to show that this function has a 
unique zero in [0,1] in case 1) or 2) holds. Indeed, if 1) or 2) holds, the sign of the derivative 
with respect to a of [Fg (a, 1 — a)] ^ — a does not change. Thus, this function is monotonic. This 
implies that Fg and Gg have unique fixed points. 

C. Proof of Theorem 


We use Schauder’s fixed point theorem |34| to prove the existence of a fixed point. We 
start by showing that Gp is a compact operator, that is continuous and maps bounded sets to 
relatively compact sets. Let x be in C'([0,T], and be a sequence converging to x 

in (C'([0,T],M’"), ||.||oo). Let 


Qj > max + max 


(6»,t)eex[o,T]2 


(6»,t)eex[o,r]4 


+ max||M® 
eee 
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We have 

\\Gp{xk)-Gp{x)\\^<J2Q. 

where 


' .*11 I K I + . 

- \\Xk ~ 3:||oo + ^Ij H-^2j f ; 


i=i 



fj 

0 jR" 



(I 

0 JR" 


Under Assumption 




- j 

7 

0 JR" 

lofl (x°)-lo0 (a:°) 

Dj(x„) Djix) 

But, 

1 o9 

o 

1 

o 

o 

VI 

o 


Dj(xk) 

D,(x) 


and converges to zero for all 

(x°, 9) in M"- X 0. We have E| a 


dPodPe. 


\x^\\dPodPe. 


dominated convergence theorem |^|, Vij converges to zero. By the same technique, we prove 
that V 2 j converges to zero. Hence, Gp is continuous. Let L be a bounded subset of C([0, T], M”). 
Let {Gp{xk)}k&N e Gp{V). By the continuity of $j(cr, t) with respect to (cr, t, 9), of its derivative 
with respect to t and a, and by the boundedness of Xk, one can prove that for all (f, s) in [0, T]^, 

\\Gp{xk){t) - Gp(£fc)(s)|| < + K 2 ^\t - s|, 

where Ki and K 2 are positive constants. This inequality implies the uniform boundedness and 


equicontinuity of {Gp{xk)}k&N- By Arzela-Ascoli Theorem [34|, there exists a convergent sub¬ 
sequence of {G'p(xfc)}fceN- Hence, Gp{V) and its closure are compact sets, and Gp is a compact 
operator. Now, we construct a nonempty, bounded, closed, convex subset U C (^([O,T], 


such that Gp{U) C U. Let Q = max(fci + ^ 2 , ^ 3 ), where ki, ^2 and are defined in (36). We 
start by defining on [0,T] the function R{t) = Q cos{y/Qt) + Q tan(A/QT) sin(A/Qf). Under 
Assumption 4. R{t) is positive. Moreover, R satisfies R{t) = Q + Q /J R{t) drda. Let 

U = |x e C'([0,T],M”)| ||x(f)|| < R{t), Vf e [0,T]|. 

The set U is an nonempty, bounded, closed and convex subset of (^([O,T],M"). For all x eU, 
Vf e [o,T], 


\\Gp{x)m < Q+Q 


R{t) drda = R(t). 


Hence, Gp{U) C U. By Schauder’s Theorem, Gp has a fixed point in U. 
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Appendix C 
Proof of Theorem 

We consider an arbitrary agent i G {1,iV} applying an arbitrary full state feedback control 
law Ui- Suppose that this agent i can profit by a unilateral deviation from the decentralized 
strategies. This means that 

Ji{ui,u%) < Ji{u*,u%). (39) 






In the following, we prove that this profit is bounded by e. We denote respectively by Xi and 
the states corresponding to Ui and u*. In view of (30), the compactness of 0, the continuity of 
with respect to 6 and E||a;°p < oo, the right hand side of (39) is bounded by Qi independently 
of N. For any X and Y in (7([0, T], M"^), we define 

< X\Y >=e(^J^ X'^{t)Y{t)dt x^i 
and ||X ||2 = < X\X >. We have 


1 ^ 2 

Ji{ui,uY) = Ji(^Xi{ui),x,x°^ +1 X - — ^x] ^ + - XiWl + Si + S2 + S3, 


where 




X, 




1 ^ 


N 

j=i 


N 




Xi- X), 


i=i 


with a: is a fixed point of Gp. By the Cauchy-Schwarz inequality. 




2 OC 




In view of (39) and the bound Qi, 


Xi 

and 

Xi-X 


2 



are bounded. Thus, \Si\ <rii/N, 


where r^i > 0. Similarly, |S' 2 | <'r] 2 /N, where r ]2 > 0. We define 


aN — 


1 ^ 

X -—Ex* 


x^ dPe 


'e 


'e 


X^ dPg^ 
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where is defined in (|35|). We have 

2 


N 


N 


N ^ ^ 

i=i 


< 2a + 2 


N 






i=i 


By the eompaetness of [0,T] x 0, the family of funetions x^{t) defined on 0 and indexed by t 
is uniformly bounded and equieontinuous. By Corollary 1.1.5 of [(43, we deduee 


N 


lim sup 

A1-S-+00 tg[o,r] 




i=i 


= 0 . 


Thus, Oat eonverges to 0 as iV inereases to infinity. By the independenee of the initial eonditions 
(and thus the independenee of x*, j = 1,..., A^) and the assumption E||a;°|p < cxd, we deduee 
that 


1 

iv 


N 


E - 




1=1 


= 0(l/iV). 


Thus, S'a and 
we have J, (x, 


eonverge to 0 as iV inereases to infinity. By optimality. 


Ji[x*,x,x^ ) — Ji(u*, u*_^ ) + S*! + 52 + 53 eonverges to 0 as iV inereases to infinity. 


Ji(^x*,x,x^^-Ji(u*,u*_, 

,f, < Ji(^Xi,x,x^'^. Therefore, Ji{ui,u*_j) > Ji{u*,u*_i) + e^v, where e^v = 
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